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Canonical Analysis as a Generalized Regression T ^:hn^Que for 
Multivariate Analysis 

John D. Williams 
The University of North Dakota 

The use of characteristic coding (dumny coding) is made in showing 
solutions to four multivariate problems using canonical analysis* The 
canonical variates ^^ji ue analyzed by the use of multiple linear regression. 
When the canonical variates are used as criteria in a multiple linear 
regression, the values are equal to where ^3* is the squared canonical 
correlation coefficient* Several different methods exist for testing 
multivariate hypotheses* Where the interest is in a two-way disproportionate 
multivariate analysis of variance, the trace criterion {l^^} seems particularly 
applicable* 

Characteristic (dumniy) coding has been used in multiple linear regression 
to analyze univariate analysis of variance problems; the same coding scheme 
can be ex-tended to multiple criteria* While the resulting data are analyzed 
through canonical analysis, the design matrix conforms to the usual multiple 
linear regression design matrices. Thus, the utilization of multiple 
criteria can be pursued in a logical sequence without necessitating con^* 
tinuously changing the entire terminology. 

In the present paper, four multivariate research designs are examined 

in a canonical analysis framework: a multivariate two-group situation^ 

2 

sometimes referred to as Hotelling's T test; a multivariate multiple group 
situation; a multivariate two-way analysis; and a multivariate two-way 
analysis with disproportionate cell frequencies* 

Tests of Significance in Canonical Analysis 

In multivariate analysis, several different tests of significance 
are used. Typically, the multivariate analysis of variance has focused 
on solving the following equation for 
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whereas canonical analysis has focused on solving 

h2S'22^21 " 0 
2 

Where -0-= and-ftj = ; also, 

^11' ^12' ^21 ^ZZ Variance ^ covariance matrices, 

Roy's (1957) largest root criterion tests the significance of the largest 
characteristic root, Hotelling's (1951) trace criterion tests the overall 
multivariate hypotheses for all dimensions simultaneously and is given 
by trace " Z\y Tables for testing either of these two tests have been 
given by Pillal (1950)* A trace criterion using Pillai's tables wherein 
the trace is 2^^, is useful In testing the significance of the overall 
set of canonical correlations and is analogous to Hotelling's trace 
criterion- 

Wilks A also provides a test of the overall hypothesis. All 
necessary tables for testing these hypotheses can be found in Timm (1975)* 
Typically, in canonical analysis simultaneous tests of each characteristic 
root using Roy's approach is used* On the other hand, multivariate analysis 
of variance programs usually employ an overall test (either the trace 
criterion or A)* Harris (1975) has argued that the use of the largest root 
criterion is more sensible in that if either the trace criterion or A 
shows significance but the largest root criterion does not, then the 
differences among the groups cannot be pinpointed by any single linear 
combination of variables* Harris would see the use of the overall hypotheses 
as being more useful in only those cases where X] and ^2 are close to the 
same Value. 



ERIC 



3 



A Multivariate Two'Group Situation (Hotelling's r Test) 

The simplest multivariate analysis of variance situation is the multi- 
variate analog to the usual t testj here» several criteria are observed for 
two groups and an overall test for group d'^fferences can be made. As an 
example, suppose four criteria (Y-j , Yg, Y3 and Y^) are observed for two 
groups as indicated in Table 1. 



Table 1 

Four Criteria Y^» Yg* Y^, and Y^ 
for the Multivariate Tv/o-Group Situation 



Group 1 



Group 2 



h 


\ 


h 


Y4 


^1 


Y 

2 


^3 


^4 


20 


17 


17 


25 


15 


26 


13 


16 


22 


19 


16 


28 


19 


25 


15 


15 


24 


14 


18 


23 


23 


21 


17 


17 


26 


16 


17 


17 


24 


17 


22 


18 


28 


18 


16 


29 


25 


19 


16 


22 


30 


20 


15 


32 


■ 2'6" 


IT 


8 " 


"2'3 


32 


22 


14 


35 


27 


22 


14 


24 


34 


16 


16 


42 


28 


20 


18 


20 


36 


9 


18 


38 


30 


17 


21 


18 



To accomplish a canonical analysis with the data in Table 1, it 1s 
necessary to define a first- set and a second- set. For convenience^ the 
criteria will constitute the first set and the predictors {group membership 
variables) will constitute the second set. Actually^ only a single group 
membership variable is necessary; 

= 1 if a member of Group Onej 0 otherwise. 

Table 2 contains the criteria and design matrix necessary to accomplish 
this analysis. 
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Table 2 



Criteria and Design Matrix 
for a Two-Group Multivariate Analysis 



J 


L 


^3 


^4 


20 


17 


17 


' 25 


22 


19 


16 


28 


24 




18 


23- 


26 


16 


17 


17 


28 


18 


16 


29 


30 


20 


15 


32 


32 


22 


14 


35 


34 


16 


16 


42 


36 


9 


18 


38 


15 


26 


13 


16 


19 


25 


15 


15 


23 


21 


17 


17 


24 


17 


22 


18 


25 


19 


16 


22 


26 


14 


8 


23 


27 


22 


14 


24 


28 


20 


18 


20 


30 


17 


21 


18 



0 
0 
0 
0 
0 
0 
0 
0 
0 
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Using canonical analysis to find the relationship betiveen the left 
set and the right set, the following results are obtained; 
'9-= .60066; canonical R^. =^5"= .77502; 

Wilk's A = .39934, with p < .01. Also, the trace = .60066: in every 
case, p < .01 . 

The coefficients for the first set are 



'1 



First root 
-.65403 

Yg -.32522 - 

Y3 .24849 

Y4 1.26372. 
The coefficient for the variable is of course 1.000. 
Al so, 

T. = 26.0556, Sy = 5.3189; 



To = 18.4444, s^, = 4.0905; 

To 
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Y, - 16.1667, sv ^ 3.0534; and 
T4 = 24.5556, Sy^ = 7.9648. 
If a nei7 variable, Yg, is formed as 

Yg = -.65403(Yi - Y^j^Y^ " -^^^ZatYg - \)/^^^ + .24849(Y2 - 
+ 1.26372(Y^ - Y^VSy 

4 , 

then a regression equation can be formed as 

Yj = b^ + biXi + ei. (1) 

If a regression is completed with the formulation in eouation 1, then 
R = .77502, 

« .60066, and 
1-R^ = .39934. 

This information is identical to that found in the use of the canonical 
analysis; the relationship is, for the two-group situation: 
R =VT = R(,; 

R^ « -e-and h 1-R^. 

The use of equation 1 shows that a composite variable, Yg, is a linear composite 
of variables Y-j , Yg. Y^ and Y^ such that the relationship vHth X-] remains 
maximized. 

Because there is only one group membership variable involved, an in- 
teresting reversal of the roles of the criteria and predictor can be made: 
X| s b(, + b^Yj + bgYg + bjYj + b4r4 + eg. (2) 
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If equation 2. is utilized in a multiple regression frriework, 

R = .77502. = .60066. 1-R^ = .39934 and F 4.888. 
The first three results were obtained in the prior two analyses. 

There are some differences between the two analyses, however. For 
the use of equation 2. b, - b^ are different from the coefficients for 
the first set given previously; this, of course, was to be accepted. The 
beta coefficients also differ from the coefficients given earlier; 

&■^ = -.50688. 

^2 = -.25206. 

^ = .19259. 
= .97940. 

If some thought is given to it. this difference comes dt no surprise 
either. In a canonical analysis, each canonical variate has a mean of 
zero and standard deviation of one. In a regression analysis, the beta 
coefficients are such that for every predictor variable, there is a mean of 
zero and standard deviation of one. The difference is that in canonical 
analysis, the new variate is created with mean zero and standard deviation 
of one. 

Finally if a multivariate analysis of variance prcgran is executed. 
A= .39934 and F - 4.888. results that were obtained earlier. Thus, if 
the interest is in comparing two groups on several criteria simultaneously, 
several different strategies alloh' equivalent. solutions. In this special 
case, the execution of equation 2 (using the group membership variable 
as the criterion and the Y variables as predictors) is perhaps the 
easiest solution to employ* The use of canonical analysis and subsequent 
formation of a composite variable would also seem to be of some value* 



A Multivariate Multiple Group Situation 

If several groups are involved in the analysis with multiple criteria, 
then the usual one-way multivariate analysis of variance is often employed. 
As an example of such a situation, suppose three criteria are available 
for three grouns of subjects. Such a situation is encountered in Table 3. 



Table 3 

Criteria and Design Matrix for Multivariate Analysis 
of Variance Through Regression 



^1 






^1 
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17 


23 


1 




0 


22 


28 


.2 




0 


14 


22 


3 




0 


18 


27 


.4 




0 


29 


25 


■5 




0 


22 


32 


■6 


0 




24 


34 


8 


0 




26 


36 


10 


0 




28 


42 


12 


0 




25. 


.3.1 


lA 


0 




26 


23 


15 


0 


0 


29 


32 


16 


0 


0 


32 


29 


17 


0 


0 


35 


42 


18 


0 


n 


33 


23 


19 


0 


0 



Two group membership variables are used; 

=1 if a rnember of firoup 1; 0 otherwise, and 

= 1 if a member of firoup 2; 0 othen^ise. 
Using the data in Table 3, a canonical analysis is performed with 
the Y scores (criteria) as the first set and the X variables (predictors) 
as the second set. Several useful items are typically available from a 
canonical analysis. Either the canonical roots or canonical correlations 
(or both) will be available. For the data In Table 3, - .89286, and 
^2 -43602. The first canonical correlation isV^^« .94491, and the 
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second canonical correlation is-V% = .66032. The weights for the Y 
side and X side are: 

Y side weights 1 2 3 

1 .09556 -.01807 -1.07322 

2 -.79902 1.10739 .27948 

X side weights 1 2 

1 1.1 ME7 .56235 

2 .01725 1.00851 

It is interesting to form variables to correspond to those suggested by 

the Y side and X side weights and investigate these transformed variables using 

ordinary-multiple regression. To utilize the weights, it is first necessary 

to transform all the data in Table 1 into z scores. As X] = .3333, 

Xg = .3333, Y-j = 25.3333, = 29.9333, = 10.0, = .4880, = *4880, 

s = 6.0198» s = 6.5407 and s„ = 6.2678, the transformation equations 
yi Vz -^3 

are: 

= |T09556(Y^-25.3333>6.0198j + [^.01807(Y2-29.9333)/6.5407j + 
{[;T.07322(Y3-10.)-/6.-26783 , - 

= [-.79902(Y^-25. 3333)76.0198] + [l .10739(Y2-29.9333)/6.5407j + 
[]27938(Y3-10.)/6.2678j 



Z3 = (l.l5457(X^-. 3333)/. 4880] + |^56235(X2-.3333)/.4880] 

Z4 =[;01725(Xi-.3333)/.4880j + fl •0085l{X2-.3333)/.488o] 

Using Z^ as the criterion and X^ and Xg as predictors^ = .89286» 
R = .54491, identically the same results as found for the first canonical 
root. Similarly » using Zg as the criterion and X^ and Xg as predictors » 
R^ = .43602 and R = .66032. If Y^ , Yg* and Y3 are used as predictors of 
Z3 and then Z^, again the canonical correlations appear as multiple correlations, 



Also the following correlations are of interest: 



r? 7 = 0 
12 

1 c 


r =0 
Z2Z3 


r, 7 = .94491 


r, , = .66032 


r. - = 0 


^3^4 


If a traditional multivariate analysis of variance is oerfonned, the 



test for (overall difference among all groups) yields A - *n6042* 

While some canonical printouts (such as Cooley and Lohnes, 1971) include 

this value as part of the output, A can be found as (l-&^) where the 

fl* are the canonical roots* For this particular data, A - (1-»89286)(1 -,43502) 

*06042* Because-fr, = R?, this result can be written as A n*(l-R?)* The 

trace criterion yields Tr = *89286 + *43602 = 1.2888. 

Also, from the multivariate analysis printout, an F = 10*227 (p. <*01) 
tests the overall group differences among the three groups. 

The use of the canonical variates found through the use of the canonical 
vectors should Present an attractive alternative to those researchers who 
wish to complete multiple conparisons after the rejection of the overall 
null hypothesis* One suggestion, made by Huntmel and Sligo (1971) is to 
compare the groups on univariate tests on each variable after the rejection 
of the overall null hypothesis* An alternative is to use the first 
canonical variate for the criteria set and run an analysis among the groups 
with this (and subsequent) canonical variates from the criteria set* Scheff6's 
test would seem appropriate as a multiple comparison method* 
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A Two-Way Multivariate Situation 

The two-way multivariate analysis of variance is not quite as available 
as the one-way multivariate analysis of variance, but several programs 
are available, including Cramer (1974), Finn (1974), Ondrack (1974), and 
Cooley and Lohnes (1971). Suppose three criteria are measured in a 2X3 
design. The design matrix is given in Table 4, 

Table 4 



Criteria and Design Matrix For Two-Way 
Analysis of Variance Through Regression 
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15 


24 
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20 


13 


22 
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0 
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0 
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22 
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18 
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0 
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0 
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17 


15 


16 


0 . 
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0 
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18 


17 


15 
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0 
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0 
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19 


19 


14 
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0 
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0 
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20 


21 


14 
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0 




0 
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0 


21 


23 


18 
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0 
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0 
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22 


25 


19 
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13 


17 


22 


0 


0 


0 


0 
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0 
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22 
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(J 
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where 



for a member of Row 1 and Column 1, 0 otherwise; 




for a member of Row 1 and Column 2, 0 otherv/ise; 



for a member of Row 1 and Column 3, 0 otherwise; 



for a member of Row 2 and Column 1 , 0 otherv/ise; 



Xg - 1 for a member of Row 2 and Column 2, 0 otherwise; 

Xg - 1 for a member of Row 1 , 0 otherwise; 

X^ = 1 for a inember of Column 1, 0 othen-zise and 

Xg = 1 for a member of Column 2, 0 otherwise. 

An analysis of the data in Table 5 is necessarily complex. Four 
sets of canonical relationships are possible; the Y-) , Y2 and variables 
can be related to X^ - X5, then Xg and X7, then Xg and finally Xg, X7 and 
In the univariate situation these relationships correspond to the full 
model, the rows, the columns and the roivs and columns as predictors. 
Table 5 contains the various canonical relationships from these four 
different sets of predictors* 
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Table 5 



Eigenvalues, Canonical Correlations and Wilks Lambda For 
Two-Way llultnariate Analysis of Variance by Regression 



Preditors: X^, Xg, X^, X^, Xg (Full Model) 



First Root 
Second Root 
Third Root 



.58716 
.35345 



Canonical R 
.93024 
.76626 
.59452 



.03594 ** 
.26692 ** 
.64655 ** 



Predictors: Xg (Rows) 
First Root .32400 



.56921 



.65600 ** 



Predictors: X^, Xg (Columns) 



First Root 
Second Root 



.45924 
.25968 



.67767 
.50959 



.40033 ** 
.74032 



Predictors: Xg, X^, Xg (Rows and Columns) 



First Root 
Second Root 
Third Root 



. 54326 
.30626 
.19340 



.73706 
.55341 
.43977 



** significant at .01 level 



.25558 ** 
.55957 ** 
.80660 ** 



Corresponding to each canonical root are the weights to create the canonical 
variables^ they can be found (for the Y side only) in Table 6. 
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Table 6 

Canonical Weights for Canonical Variates from Table 5 
Predictors; X-) - Xg 



Variste 
h 



Y-Weights 



1 



-.44831 
-.85876 
.40964 



2 

-.?0691 
-.05983 
•1.02427 



3 

.73004 
-.68227 
■.23006 



Predictor; Xg 

Variate 
24 

Predictors; X^, Xg 



.16443 



.62240 
-.62870 



■.45391 



.65303 
.69649 



.85726 



.10464 
.59305 



Predictors; Xg, X^, Xg 
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-.46410 
.00110 

.94386 



-.62886 
-.58279 
-.60035 



.32592 
-.91247 
.33548 



If - Zg are used as criteria in a multiple linear regression 
layout* then v;hen X-^ - Xg are used as predictors of Z^, R = .86535 « 0*1 
similar findings will occur with the Z variates that correspond to the 
X predictors used in the original canonical analysis. If the trace 
(sum of the squared canonical correlations) of the four models are found 

Tr (Full) = .86535 + .58716 + .35345 = 1.80596 

Tr (Rows) = .32400 

Tr (Columns) « .44924 + .25968 - .71892 



Tr (Rows and Columns) = .54326 + .30626 + .19340 = 1.04292 - 

al so, 

Tr (Rows) + Tr (Columns) = Tr (Rows & Columns) . 
The interaction can be defined as 

Tr (Full) - Tr (Rows & Columns) 1.80596 - 1.04292 = .76304. 

The sum of the squared canonical correlations can be broken down into 

2 • • 

the separate values through the use of orthogonal coefficients, as 

there are an equal number of entries in each of the six cells. If five 

new variables are defined as follov;s 

Xq = 1 if a member of Row 1, -1 if a member of Row 2; • 

X-jo = 1 if a member of Column 1, 0 if a member of Column 2, -1 if 

a member of Columrv 3-; 

X^^ « 1 if a member of either Column 1 or Column 3, -2 if a member 

of Column 2; 

Xi2 ~ Xo'X-jQ and 

X^3 Xg-Xii 

2 

Using X^ as the predictor of the three criteria, = .32400, the same 

result as found in Table 6 for rows. When X^q and Xii are used as predictors. 
2 ? 

R^ = ,45924 and R = .25968, the same result as found in Table 5 for 

^ '2 2 

columns. If X^, X^p, X^^, X^g and X^^ are used as predictors, ~ ,86535, 

R^ = .58716 and K = .35345, the same results as found for the full 
^2 ^^3 
model. If Xg, X-jq and X-j^ were used as predictors, the results would 

duplicate those found by usinn Xg, and Xg as predictors. If X^g 
and X^2 are used as predictors, the fallowing results are found: 



R^^ « ,42305 R^^ = ,65042 = .38079, p ^ .01; 

rJ = ,33999 R-:, = ,58309 = .66001, p <,01, 
C2 ^2 ^ 



and 
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Finding the interaction directly through the use of orthogonal polynomials 
appear*: to be limited to those cases in which the cell entries are either 
equal or proportional. The last problem to be discussed considers the 
multivariate disproportional case. 

A Two-Way Disproportionate Multivariate Analysis 

An analysis similar to the one employed for the two-way equal cell 
case shown in Table 4 can be considered. In fact» the same 36 "subjects" 
are reconsidered, after deliberately creating a disproportionate situation. 
The first 3 subjects are, for the disproportionate case, in cell 1 (Row 1 and 
Column 1); the next four subjects are in Cell 2 (Row 1 and Column 2); 
the next 10 scores are in Cell 3 (Row 1 and Column 3; the next 9 subjects 
are in Cell 4 (Row 2 and Column 1); the next 7 subjects are in Cell 5 
(Row 2 and Column 2); finally, the last 3 subjects are in Cell 6 (Row 2 and 
Column 3). The number of entries in each cell for the 2X3 layout is as 
given in Table 7. 

Table 7 

Frequencies for 2X3 Multivariate Analysis with 
Disproportionate Cells 

Column 1 Column 2 Column 3 

Row 1 3 4 10 

Row 2 9 7 3 

The design matrix is as before with X^ - Xg having the same meaning. 

The results of the canonical analysis are found in Table 8, 
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Table 8 



Eigenvalues, Canonical Correlations and Wilks Lambda for 
Two-Way Disproportionate Cell Frcauencies 



Predictors: X^. Xg. X3. X^. Xj (Full Model) 



First Root 
Second Root 
Third Root 



.61960 
.31954 
.12824 



Canonical P. 

.78715 
.56528 
.35811 



. 22565** 
.59320* 
.871 76 



Predictors: Xg (Rows) 

First Root .27262 



.52213 



.72738* 



Predictors: X^, Xg (Columns) 



First Root 
Second Root 



.21675 
.02601 



.46557 
.16127 



: 762 87 
. 97399 



Predictors: Xg, X7, Xg (Rows and Columns) 

First Root .34802 
Second Root .21674 
Third Root .02559 
*significant at .05 level 
**significant at .01 level 



.58993 
..46555 
,.15998 



.49760** 

.76322 

.97441 



Interpretation of the data in Table 8 may be made, but the lack of 
consensus on interpreting univariate disproportionate situations will only 
be increased as the situation becomes multivariate. Many authors prefer 
the "fitting constants" solution (see Anderson & Bancroft, 1952, Overall 
and Spiegel, 1969^ and Rao, 1965). Cohen (1968) describes a partioning 
solution called the hierarchical model. An unadjusted main effects 
solution is shown in Williams (1972). Searle (1971) and Appelbaum and 
Crancr (1974) prefer a multiple step decision making process that combines 
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the fitting constants solution and the unadjusted main effects solution. 
The multivariate situation is complicated by the existence of several 
criteria for judging the significance of an experiment. The approach taken 
here is to describe both the fitting constants solution and the unad- 
justed main effects solution; those who prefer the decision rules given 
in Searle could easily employ them with the information given. 

The Unadjusted Main Effects Solution 

The unadjusted main effects solution follows in a manner very similar 
to the one presented in regard to the equal cell case. In fact, the data 
in Table 8 can be interpreted (except for the interaction) as beirig 
an unadjusted main effects solution. The interaction can be found as 
the difference between the trace of the full model and the trace of the 
rows and columns model: 

(,61960 + ,31954 + ,12824) - (,34802 + ,21674 + ,02559) = ,47703, 
The only available method to test the interaction hypothesis is Pillai's 
trace criterion; p < ,05, 

A Multivariate Analog to the Fitting Constants Solution 

Because of the disproportionality of the data, the direct calculation 

of the terms is precluded; the traces can be found in a manner similar 

to finding the trace for the interaction, however. The trace for rows 

(after removing the effect for columns) can be found as the trace for rov^s 

and columns niinus the trace for columns; 

(,34802 + ,21674 + ,02559) - (,21675 + ,02601) = ,34759, p <,01, 

The trace for columns can be found as the trace for rows and columns minus 

the trace for rows: 

(,34802 + ,21674 + .02559) - (.27262) = ,31773. p >,05, 

The interaction is the same as given for the unadjusted main effects model, 
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Discussion 

While Harris*^ argument for the use of the greatest characteristic root 
criterion as a measure for multivariate analysis is noteworthy^ the trace 
criterion (iB^) is particularly useful with disproportionate cell frequencies. 
If the focus of the greatest characteristic root and the corresponding canonical 
variates are made on the full model (cell modeU or full rank model) then 
either criterion is applicable^ and perhaps Harris's suggestion is appropriate. 
If the intent is on producing a two-way MANOVA with disproportionate cells 
and there is interest in the row> column and interactions effects^ then 
Pillai's trace criterion is most appropriate. Even where there is 
interest in the usual effects > the most likely canonical variate to be 
of interest is the variate associate with the greatest characteristic root 
from the full model. 

Four different multivariate applications have been shown herein. 
Other applications (multivariate trend analysis^ multivariate analysis of 
covariance and other analogs to univariate designs) are possible through a 
canonical approach. Also> the univariate analyses that can be performed 
by multiple linear regression can be conceptualized as a canonical problem. 
That the canonical analyses and multiple regression analyses would yield 
quite similar results is not quite the same as saying that the analyses 

are identical for the univariate situation. As was shown in Hotelling's 

2 

T test> some differences in weighting coefficients occur. The overall 
results {R^*s) are identical > however. 
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